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(54) Video surveillance system 

(57) A video surveillance system for monitoring an 
area is made up of a stationary-pilot camera ( 1 2) for mon- 
itoring the area. and one or mqre moveable slave, cam- 
eras (16,18,20,22) for monitoring at least part of the 
area. Each slave camera is aNopated to part of the area 
being monitored. The pilot camera produces a signal 
representative of the area. The location of a moving 
object in the area monitored by the pilot camera is deter- 
mined. A signal is produced representing the location of 
the object. The slave cameras track the object based on 



the signal representing the location of the object. The 
pilot camera is provided with a f isheye lens and thus has 
a fielci of view wider than that of the slave cameras. The 
area being monitored is divided into cartesian, polar or 
spherical coordinates thereby enabling a microcomputer 
to instruct the slave camera to follow the object. The sig- 
nal representative of the area is compressed and trans- 
mitted over a communications channel for remote 
monitoring. 
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Description 
BACKGROUND 

The present invention relates generally to video sur- 5 
veillance systems. More particularly, it relates to a video 
surveillance system in which a pilot camera controls the 
operation of one or more slave cameras. 

Known video surveillance systems often monitor rel- 
atively large areas by means of several video cameras 10 
or "domes." A dome may include one or more cameras, 
the operation of which can be remotely controlled. A 
camera in such a dome is typically equipped with a zoom 
lens which has a limited field of view. The camera can 
also be panned and tilted by remote control. The extent 15 
to which a particular camera or dome is able to monitor 
an area is limited by the field of view of its lens and the 
extent to which it can be panned and tilted. It is therefore 
possible for an intruder to avoid detection by a surveil- 
lance system if the can dodge the movement of the cam- 20 
era and keep out of its field of view. Thus, in order to 
minimize this problem, particularly when the monitored 
area is large, a system of the prior art uses a large 
number of domes operating under program control to 
cover the entire area. However, such systems do not 25 
eliminate the risk that an intruder will be undetected. In 
addition, such systems require a large number of cam- 
eras and a relatively complex control program in order to 
minimize the opportunity for intruders to escape detec- 
tion. 30 

SUMMARY OF THE INVENTION 

The present invention is a video surveillance system 
for monitoring an area. The system is made up of a pilot 35 
camera for monitoring the area and a slave camera for 
monitoring at least part of the area. The pilot camera pro- 
duces a signal representative of the area. Means are pro- 
vided for determining the location of a moving object in 
the area monitored by the pilot camera from the signal 40 
representative of the area and for producing a signal rep- 
resenting the location of the object. Means are also pro- 
vided for causing the slave camera to track the moving 
object based on the signal representing the location of 
the object. 45 

In the preferred embodiment the pilot camera is sta- 
tionary and the slave camera is moveable. The pilot cam- 
era has a field of view wider than that of the slave camera. 
To that end the pilot camera is provided with a fisheye 
lens. The signal containing the fisheye image produced 50 
by the camera can be compressed and transmitted over 
a communications channel to a remote monitoring sta- 
tion. The location of the object is determined by detecting 
the motion of the object. 

The invention also includes a method of monitoring 55 
an area comprising the steps of: monitoring the area by 
means of a pilot camera, thereby producing a signal rep- 
resentative of the area; processing the video output to 
determine the location of a moving object in the area and 
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producing a signal representative of the location of the 
object; and controlling a slave camera such that the slave 
camera tracks the moving object based on the signal rep- 
resentative of the location of the object. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a perspective view of a space monitored by 

the present invention, showing a pilot camera and 

slave cameras as well as the field of view of the pilot 

camera and one slave camera; 

Fig. 2 is a diagram of the image produced by the pilot 

camera; 

Fig. 3 is a block diagram of the control system of the 
present invention. 

DETAILED DESCRIPTION 

The following is a description of the preferred 
embodiment of the invention. It is intended to be illustra- 
tive of the invention rather than limiting. The full scope of 
the invention is to be determined by the appended claims 
and their equivalents. 

In Fig. t , the invention is shown installed in a large 
room 1 0 such as a warehouse having ceiling 1 4, floor 24 
and walls 34. A stationary pilot camera 12 is contained 
in pilot dome 1 1 mounted on ceiling 14. Pilot camera 12 
is a video camera of a type commonly used in video sur- 
veillance, except that it has a lens 13 (see Fig. 3) having 
a very wide angle of view, preferably a fisheye lens hav- 
ing a field of view of 180 degrees. The field of view of 
lens 1 3 of pilot camera 1 2 is wide enough for it to monitor 
an area 15 corresponding to the hatched area in Fig 1 
and covering the entire expanse of floor 24 and part of 
walls 34. 

Slave cameras 16, 18, 20 and 22 are mounted on 
ceiling 14. Each of slave cameras 16; 18, 20 and 22 is a 
conventional video surveillance camera contained in a 
"dome." The term "dome" refers here to an enclosure 
housing at least one camera capable of panning, tilting 
and zooming. The preferred dome is the SpeedDome 
available from Sensormatic Electronics Corp. of Deer- 
field Beach, Florida. Each slave camera 16, 18. 20 and 
22 can be tilted, panned and zoomed by remote control. 
Cameras 16, 18, 20 and 22 respectively have lenses 17, 
19,21 and 23 (See Fig. 3) . Lenses 17,19,21 and 23 are 
preferably zoom lenses which have a field of view less 
than that of lens 13 of the pilot camera (ranging from 6 
to 60 degrees in a 1 0X lens). Each slave camera is allo- 
cated a portion of the total field of pilot camera 12. Each 
camera 16, 18, 20 and 22 thus monitors part of the total 
area 15 monitored by pilot camera 12. Cameras 16, 18. 
20 and 22 are respectively assigned to monitor cells 26, 
28, 30 and 32 which are formed by dividing floor 24 and 
walls 34 into sectors as shown in Figs. 1 and 2. While 
the field of view of lenses 17, 19, 21 and 23 of cameras 
16, 18. 20 and 22 is far narrower than the full extent of 
each cell, the cameras can pan. tilt and zoom so each 
camera is able to cover the cell to which it is assigned 



EP 0 714 081 A1 



2 



3 



EP 0 714 081 A1 



4 



(i.e. a part of the total area monitored by the pilot cam- 
era). The ability to pan and tilt allows each slave camera 
to track the movement of an object (an intruder for exam- 
ple) within the cell to which the camera is assigned and 
into an adjacent cell. 

The invention is illustrated in more detail in blockdia- 
gram form in Fig. 3. The system comprises a pilot dome 
11 containing a pilot camera 12 having' fisheye lens 13 
and^slave cameras 16- 18, 20^and 22. respectively hav- 
inglenses V7, 19, 21 and 23: Such cameras are conven- 
tional PTZ camera devices available from Sensormatic 
VI Electronics Corp. of 'Deerf ield'Beach;- Florida under the 

narne Sp'el§ciDbme. SUch a camera incorporates a con- 
troller in the form of ZT80processor. PiTotcamera 12 and 
slave cameras 16, 18, 20*and*22 areconneeted to video 
switcher 40 which has video outputs 44. Switcher 40 is 
controlled by computer 41 via control line 43. Video out- 
put 44 of switcher 40 is fed into computer 41, which is 
provided with graphics adapter 42. Computer 41 can 
instruct switcher 40 to select any of cameras 12, 16, 18, 
20 and 22, the output of which can be displayed on mon- 
itor 54 with or without overlaid graphics provided by 
graphics adapter 42. Video output 44 can also be con- 
nected to video cassette recorder 62 and monitor 63 as 
shown in Fig. 3. 

Computer 41 is programmed to instruct, via bus 45, 
any of cameras 1 6, 18, 20 ahd : 22 to pan, tilt, zoom, focus, 
change f-stop etc. Slave cameras 16, 18, 20 and 22 are 
respectively equipped with controllers 25, 27, 29 and 31 
which cause the camera to pan. tilt and zoom under the 
control of computer 41 . A user input 56 in the form 
of "Touchtracker" input device (available* from Sensor- 
matic Electronics) or a mouse or trackball allows the user 
to control the dperatibnof the slave cameras as well as 
switcher 40: An' example-bfa system having-the ability 
to control a plurality -of cameras from an input device and 
to overlay graphics is the Videomanager System availa- 
ble from Sensormatic Electronics. 

Fig. 2 represents the image 60 produced by fisheye 
lens 13 of pilot camera 1 2. Pilot camera 1 2 is placed ver- 
tically above floor 24. Due to the fact that lens 1 3 is pref- 
erably a fisheye lens which has an extremely wide angle 
of view (180 degrees), camera 12 is able to monitor all 
of cells 26, 28, 30 and 32. The pilot camera generates 
an electrical signal representative of the area it monitors 
and reproduces the scene within its field of view as a 
circular and flat fisheye image 60 as shown in Fig. 2. In 
spite of the fact that walls 34 are perpendicular to floor 
24, the fisheye lens causes walls 34 to appear flat but 
somewhat distorted at the outer edges of the image pro- 
duced by camera 12. 

Any point in cells 26, 28, 30 and 32 can be described 
by reference to a cartesian (or polar/spherical) coordi- 
nate system. Image 60 is made up of a large number of 
pixels. Therefore, each pixel can be mapped to, and is 
identified by, a reference to the chosen coordinate sys- 
tem. A map of the coordinates versus the pixels in the 
image generated by camera 12 is stored in the memory 
of computer 41 . Thus, for substantially every pixel corre- 



sponding to cells 26, 28. 30 and 32 within image 60, there 
is a unique coordinate stored in the memory of computer 
41. The location of an object in any of the cells can be 
described by reference to the coordinates corresponding 

5 to the location. Computer 41 is programmed to instruct 
contrpaers.25, 27, 2,9.and31 respectively tadirect slave 
carTjeras 16. 18, 20 and 22 to point at a particular point 
in tfte cell to which Xfye particular slave camera is 
assigned, That poirjtis i<J^r^lfi!§d I by coordinates stpred 

10 »n the memory of compute! 41- and corresponds to spe- 
cific pixels in image, 60. Thus, for exapple. if an object 
70 -(in this case an intruder,) at a. location in ceil 28 is 
detktfedj^ tg, siave. pam^ra 18 

can'be.:directed by computer 41 to. pan ancj/or tilt and 

75 thys pojnt- aMhe locatjpfi and to follow ,th§ movement of 
an object 70 within the cell to which it is assigned. Cam- 
era 18 tracks object 70 at least until object 70 moves into 
an adjacent cell 30, when computer 41 instructs the cam- 
era assigned to that cell (camera 20) to take over the 

20 surveillance from camera 18. In an alternative embodi- 
ment, morethan one of slave cameras 1 6, 1 8, 20 and 22 
is instructed to track object 70 so that it can be viewed 
from several angles. 

In order to control a slave camera such that it tracks 

25 the motion of an object such as an intruder, it is neces- 
sary to locate the object within the area monitored by4he 
pilot camera. This is preferably accomplished by a simple 
differencealgorithm.. Successive frames acquired byatbe 
pilot cameraiare stored and subtracted from one another 

30 by storing the. previous frame in RAM (frame store) 58 
and subtracting it from the current frame pixel by pixel. 
Since the surroundings in which object 70 is found are 
substantially stationary, the subtraction of successive 
frames will cancel ouMhe surroundings, leaving only the 

35 moving object. The image remaini ng.after subtraction of 
subsequent frames is referred to as a "delta image. " This 
delta image, made up of a.number of pixels, is continu- 
ously mapped to the coordinates stored in the memory 
of computer 41 which instructs the slave camera 

40 assigned to the cell in which the object is found to point 
at the object. 

In order to account for relatively slow moving sur- 
roundings such as waving flags, plants etc.. the delta 
image can be averaged over a number of frames or cal- 

45 culated from several previous frames (e.g. four previous 
frames). If the difference indicated by the delta image 
remains within a certain maximum movement or travel 
range, it can be ignored. If the difference indicated by the 
delta image is outside the maximum movement or range, 

so a start signal will be generated so that computer 41 
instructs the appropriate slave camera to track the move- 
ment of the object. In addition, video cassette recorder 
(VCR) 62 can be instructed to start recording on such 
detection of motion. A storage device such as a disk drive 

55 can be substituted for VCR 62. Thus, as any particular 
slave camera follows the movement of an object, the 
image so acquired is stored by the VCR. As the object 
moves through successive cells, the output of the slave 
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camera assigned to the cell in which the object is found 
is fed into the VCR. 

An alternative way of detecting the location of object 
70 is to use a pilot camera having a charge coupled 
device (CCD) sensitive to specific frequencies of radia- 5 
tion emitted by object 70. for example, radiation in the 
infra-red spectrum. The slave cameras are then directed 
to the coordinates which correspond to the pixels of the 
CCD which have detected the specific infra-red radiation. 

The use of a fisheye lens in the present invention w 
makes the image captured by camera 1 2 highly suitable 
for compression and transmitted over a communications 
channel. While an object in the field of view of a fisheye 
lens moves, the background remains substantially still. 
This high temporal redundancy means that compression is 
is more readily accomplished than would be the case 
with a conventional PTZ camera since successive 
frames contain more information in common with each 
other. In order to track a moving object with a conven- 
tional camera, the camera must be panned, tilted and/or 20 
zoomed. The image changes a great deal over time as 
the camera is panned, tilted or zoomed since the back- 
ground appears to move as the camera moves. This 
makes compression more cumbersome than in the case 
of the image generated by a camera having a fisheye 25 
lens. 

Compression hardware 64 is connected directly to 
pilot dome 1 1 and to computer 41. This allows the raw 
video output of pilot camera 12 to be compressed. Com- 
pression hardware 64 is based upon a video compres- 30 
sion set available from Integrated Information 
Technology. Inc. of Santa Clara, California under the 
name Vision Controller/Vision Processor or equivalent 
circuitry. Compression schemes such as JPEG. MPEG 
or H. 261 can be implemented using this hardware. The 35 
compressed video signal produced by compression 
hardware 64 is fed into communications interface 66. 
Communications interface 66 is a conventional interlace 
such as a modem or ISDN interface. Both compression 
hardware 64 and communications interface 66 are con- 40 
trolled by computer 41 . 

The above structure allows a large area to be mon- 
itored by the pilot camera and transmitted to a remote 
monitoring station over a communications channel hav- 
ing a relatively limited bandwidth. Computer 41 instructs 45 
compression hardware 64 to compress the video signal 
generated by camera 12 and instructs communications 
interface 66 to commence transmission of the com- 
pressed video image over communications channel 68 
to remote monitoring station 72. The image is decom- 50 
pressed and displayed at the monitoring station. The 
decompressed image can then be monitored for intrud- 
ers. While the decompressed fisheye image may not pro- 
vide resolution sufficient to detect details such as the 
facial features of an intruder, when an intruder is 55 
observed appropriate action can be taken at the moni- 
toring station, for example, calling the police. In the 
meantime, computer 41 can instruct slave cameras 16, 



18, 20 and 22 to track the intruder and VCR 62 to record 
so that the intruder can be identified. 

Claims 

1 . A video surveillance system for monitoring an area, 
comprising: 

a pilot camera for monitoring the area and 
producing a signal representative of the area; 

a slave camera for monitoring at least part of 
the area; 

means for determining the location of an 
object in the area monitored by the pilot camera from 
the signal representative of the area and for produc- 
ing a signal representing the location of the object; 

means for causing the slave camera to track 
the object based on the signal representing the loca- 
tion of the object. 

2. The system of claim 1 wherein the pilot camera has 
a field of view wider than the field of view of the slave 
camera. 

3. The system of claim 1 wherein the pilot camera is 
stationary and the slave camera is movable. 

4. The system of claim 3 wherein the means for caus- 
ing the slave camera to track the object comprises 
means for panning or tilting the slave camera. 

5. The system of claim 1 wherein the pilot camera has 
a fisheye lens. 

6. The system of claim 1 wherein the object is a moving 
object and the means for determining the location of 
the object comprises means for detecting the motion 
of the object. 

7. The system of claim 1 wherein the pilot camera pro- 
duces a plurality of images of the area and wherein 
the means for detecting the location of the object 
comprise means for subtracting successive images 
from each other. 

8. The system of claim 1 wherein the means for deter- 
mining the location of the object comprises means 
for detecting specific frequencies of radiation emit- 
ted by the object. 

9. The system of claim 1 further comprising means for 
compressing the signal representative of the area 
and means for transmitting the compressed signal 
over a communications channel. 

1 0. For use in a video surveillance system, a method of 
monitoring an area comprising the steps of: 

monitoring the area by means of a pilot cam- 
era thereby producing a signal representative of the 
area; 
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processing the signal representative of the 
area to determine the location of an object in the 
area and producing a signal representative of the 
location of the object; 

controlling a slave camera such that the stave 5 
camera tracks the object based on the signal repre- 
sentative of the location of the object. 

The method of claim 1 0 wherein the pilot camera has 
a wider field of view than the slave camera. 10 



21. The system of claim 18 wherein the pilot camera pro- 
duces a-plurality of images of the area and wherein 
the means for detecting the location of the object 
comprise means for subtracting successive images 
from each other. 

22. The system of claim 18 wherein the means for deter- 
mining the location of the object comprises means 
for detecting specific frequencies of radiation emit- 
ted by the object. 



12. The method of claim 10 wherein the pilot camera is 
stationary and the slave camera is movable. 

1 3. The method of claim 1 0 wherein the step of control- 15 
ling comprises panning or tilting the slave camera. 

1 4. The method of daim 1 0 wherein the step of process- 
ing comprises a step of detecting motion in the 
object. 20 

15. The method of claim 10 wherein the signal repre- 
sentative of the area comprises a plurality of succes- 
sive images and wherein the step of processing 



16. 



17. 



23. The system of claim 18 further comprising means 
for compressing the signal representative of the area 
and means for transmitting the compressed signal 
over a communications channel. 

24. A video surveillance system for monitoring an area, 
comprising: 

a pilot camera, comprising a f isheye lens, for 
producing^ signal representative of the area; 

means for compressing the signal represent- 
ative of the area; 

means for transmitting the compressed sig- 
nal to a remote monitoring station. 



comprises a step of subtracting successive images 25 
from each other. 

The method of claim 1 0 wherein the step of process- 
ing comprises a step of detecting specific frequen- 
cies of radiation emitted by the object 30 

The method of claim 1 0 further comprising the steps 
of compressing the signal representative of the area 
and transmitting the compressed signal over a com- 
munications channel. 35 



25. The system of claim 24 further comprising: 

a slave camera for monitoring at least part of 
the area; 

means for determining the location of an 
object in the area monitored by the pilot camera from 
the signal representative of the area and for produc- 
ing a signal representing the location of the object; 

means for causing the slave camera to track 
the obj ect based on the signal representing the loca- 
tion otthe object. 



18. A video surveillance system for monitoring an area, 
comprising: 

a pilot camera for monitoring the area and 
producing a signal representative of the area, the 
pilot camera having a f isheye lens; 

a slave camera for monitoring at least part of 
the area; 

means for determing the location of an object 
in the area monitored by the pilot camera from the 
signal representative of the area and for producing 
a signal representing the location of the object; 

means for causing the slave camera to track 
the object based on the signal representing the loca- 
tion of the object by panning or tilting the slave cam- 
era. 

19. The system of claim 18 wherein the pilot camera is 
stationary and the slave camera is movable. 

20. The system of claim 1 8 wherein the means for deter- 
mining the location of the object comprises means 
for detecting motion of the object. 



26. The system of claim 24 wherein the pilot camera is 
stationary and the slave camera is movable. 

40 27. The system of claim 26 wherein the means for caus- 
ing the slave camera to track the object comprises 
means for panning or tilting the slave camera. 



28. The system of claim 25 wherein the object is a mov- 
45 ing object and the means for determining the loca- 
tion of the object comprises means for detecting the 
motion of the object. 

29. The system of claim 25 wherein the pilot camera pro- 
so duces a plurality of images of the area and wherein 

the means for detecting the location of the object 
comprise means for subtracting successive images 
from each other. 

55 30. For use in an electronic surveillance system, a 
method of monitoring an area comprising the steps 
of: 

monitoring the area by means of a pilot cam- 
era comprising a fisheye lens, thereby producing a 
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signal representative of the area; 

compressing the signal representative of the 

area; 

transmitting the compressed signal to a 
remote monitoring station. 5 

31. The method of claim 30 further comprising the steps 
of: 

processing the signal representative of the 
area to determine the location of an object in the w 
area and producing a signal representative of the 
location of the object; 

controlling a slave camera such that the slave 
camera tracks the object based on the signal repre- 
sentative of the location of the object. is 

32. The method of claim 31 wherein the pilot camera is 
stationary and the slave camera is movable. 

33. The method of claim 32 wherein the step of control- 20 
ling comprises panning or tilting the slave camera. 

34. The method of claim 31 wherein the object is a mov- 
ing object and the step of determining the location 

of the object comprises detecting the motion of the 25 
object. 

35. The method of claim 31 wherein the signal repre- 
sentative of the area comprises a plurality of images 

of the area and wherein the step of detecting the 30 
location of the object comprises subtracting succes- 
sive images from each other. 

36. The system of claim 1 wherein the step of determin- 
ing the location of the object comprises the step of 35 
detecting specific frequencies of radiation emitted 

by the object. 
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